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ABSTRACT 

Based on the assumption that wellbeing, positive emotions and engagement influence 
motivation for learning, the aim of this paper is to provide insight into students’ 
emotional responses to and engagement in different learning designs. By comparing 
students ’ reports on the experiential qualities of three different learning designs, their 
respective influence on students’ motivation for learning is discussed with the purpose 
of exploring the relationship between positive emotions, engagement and intrinsic 
motivation for learning. Our study thus aims at evaluating the motivational elements in 
the three learning designs. This experimental, controlled comparison study was 
conducted in an introductory computer programming course. The three learning 
designs were: 1. A traditional teacher-led course; 2. A problem based learning (PBL) 
course; and 3. A PBL course combined with the use of LEGO Mindstorms Robots. 

Three different methods were used for collecting data on the students ’ experiences and 
feelings: 1. A questionnaire survey with 229 students from groups exposed to the three 
different learning designs; 2. Six qualitative walk-alongs collecting data from these 
groups by informal interviews and observations; 3. Six class room observations. 
Findings from the three studies were discussed in three focus group interviews with 10 
students from each learning design in order to validate these findings. 

The research was conducted among first year students in Computer Science at the 
Informatics School, Universidad Nacional de Costa Rica. 
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SECTION I: INTRODUCTION 

With the global increase of university students, failure rates have become a worldwide 
concern. This is also the case with the retention of first year students in Computer Science 
(O’Kelly & Gibson, 2006). Specifically, programming courses are generally regarded as 
difficult, and often have the highest failure/dropout rates (Robins, Rountree, & Rountree, 
2003). The Informatics School at Universidad Nacional in Costa Rica is no exception. In the 
period between 2008 and 2012 the average failure rate (including dropouts) of the 
introductory programming course was 47.2%. 

This increase in failure rate has generated interest in identifying factors affecting success in an 
introductory computer-programming course. A study by Wilson & Shrock (2001) examined 
whether factors such as math background, gender, previous programming experience, 
encouragement, comfort level in the course and work style preference, have an influence on 
success. The results showed that the comfort level was the strongest influencing factor 
followed by math background. The authors emphasized the importance of providing students 
with a comfortable and non-intimidating environment that motivate them to learn thus 
pointing at the role of emotions in learning. 

Motivating students has always been a challenge. In Jenkins (2001) the author studied four 
types of motivation in computer science undergraduate students: extrinsic, intrinsic, social, 
and achievement. The results suggested that extrinsic motivation is strong, that is, a large 
number of students are motivated to study computer programming because they believe they 
will have rewards such as better opportunities in their professional life. This study also 
showed that an almost equal number of students are intrinsically motivated, meaning that they 
are really engaged in their learning process for the sake of developing skills. Moreover, the 
author pointed out, that intrinsically motivated students seemed to be more interested in 
learning in general rather than specific learning of computer programming. This study showed 
that it is not straightforward to understand and to stimulate the motivation of computer 
programming for students. 

One recent trend to make computer science courses more exciting and interesting to students 
is the use of programmable LEGO Mindstorms robots (Blank, 2006; Klassner & Anderson, 
2003; Cliburn, 2006). It is widely believed, in spite of some divergent results (Fagin & 
Merkle, 2002; McNally, Goldweber, Fagin, & Klassner, 2006), that the use of FEGO 
Mindstorms provides students with a motivating learning environment (McWhorter & 
O’Connor, 2009). Fearning strategies such as critical thinking and metacognition, required to 
effectively learn computer programming, have been shown to be related to students’ 
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motivation (Bergin, Reilly, & Traynor, 2005). Moreover, robots are well suited for 
encouraging creative problem solving because they combine technological knowledge with 
soft skills such as team skills and complex problem-solving strategies (Hees, Jeschke, Natho, 
& Pfeiffer, 2011). 

On the other hand, some authors (Hamalainen, 2004; Nuutila, Torma, & Malmi, 2005) have 
stressed that a problem-based learning approach (PBL) can contribute to motivate students 
and reduce failure and dropout rates. PBL contributes to develop students’ learning through 
teamwork skills, hands-on practice skills, problem solving skills, and project organization and 
planning skills (Kolmos, Fink, & Krogh, 2004). Due to its inductive nature, PBL is believed 
to have a strong impact on the intrinsic motivation for learning, because students can 
understand the purpose of what they are learning (Prince & Felder, 2006). In addition, the 
approach promotes active and collaborative learning, and greater student responsibility in his 
or her own learning process (Prince & Felder, 2007). 

In order to investigate further the effectiveness of using PBL and LEGO Mindstorms robots to 
influence student motivation and reduce failure and dropout rates, an experimental, controlled 
comparison study was carried out in an introductory programming course at the Universidad 
Nacional in Costa Rica. The study compared three learning designs for the introductory 
programming course: (1) a problem-based learning (PBL) design; (2) a combination of PBL 
and LEGO Mindstorms (PBL+LM) learning design; and (3) a traditional learning design 
(control group) using classical teacher-led lectures and black boards. 

The overall aim of the study was to gain insight into and evaluate the influence of the learning 
designs on students’ motivation for learning. This paper reports the results related to the 
students’ emotional response to their learning experience, thus scrutinizing the findings by 
Wilson & Shrock (2001) that a comfortable and non-intimidating environment motivates 
students. These emotional responses express the hedonic qualities of the learning environment 
and the learning designs. As such they are key factors in students’ attraction or repulsion to 
the course. Based on research in motivation (Higgins, 2006; Higgins & Scholer, 2009), this 
paper furthermore examines the impact of the learning designs on the strength of student 
motivation by also looking at these designs engaging qualities. 

The goal of our study was thus to examine whether there was a relationship between the 
students’ feelings - e.g. whether they felt happy/sad, bored/stimulated, involved/disinterested, 
nervous/safe etc. - and their intrinsic motivation for learning. This relationship is the learning 
experience, understood as the experiential value of the learning process as reported by 
students during and after this process. We have examined these student experiences by using 
the ten criteria characterizing “positive experiences”, developed by Jantzen et al. (2011). 


The remainder of this paper will proceed as follows. Section II defines experiences, their 
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learning potentials and their relation with emotions. This section also presents the ten 
experience criteria used to analyze students’ emotional response. Section III reviews and 
discusses related research on LEGO Mindstorms Robots and on Problem Based Learning. In 
section IV an overview is provided of the study methodology and the three learning designs 
utilized. Section V presents the results, whereas section VI discusses the findings. The paper 
closes with discussing the implications of our research in section VII. 


SECTION II: EXPERIENCES, EXPERIENTIAL LEARNING AND EXPERIENCE 

CRITERIA 

As pointed out by Higgins (2006) the experienced value of a product or process (e.g. learning) 
is a matter of both hedonic experiences (i.e. pleasure or pain) and engaging experiences (i.e. 
intensity of engagement). The hedonic experiences determine the direction of the motivation. 
They make products and processes seem attractive or repulsive. The force of the motivation, 
though, is a result of both hedonic and engaging experiences. Engagement thus contributes to 
the degree (i.e. the strength or intensity) in which users are motivated and feel attracted to or 
repulsed by the product or process. An experience is in our study understood as a cognitive 
awareness of physiological and emotional changes in the organism (Jantzen, 2013). These 
changes have a hedonic valence. The awareness generated in experiencing challenges existing 
cognitive structures and may lead to an increased knowledge of the self and the world. 

Implied in this definition is the coherent and dynamic character of experiences. An experience 
is coherent, because it integrates physiological, emotional and cognitive aspects. It is dynamic 
firstly because actual experiences mark a difference from previous ones and because actual 
experiences are the foundation of future experiences. In experiencing, the present is related to 
the past (as expectations to be challenged) and to the future (as formation of memory). 
Secondly experiences are dynamic by encompassing an “undergoing” and a “doing” (Dewey, 
2008). We are passively exposed to experiences: They happen to us and we respond to their 
hedonic qualities emotionally (“an undergoing”). But we are also and at the same time 
actively seeking experiences: They motivate us by engaging us (“a doing”). 

Experiences have learning potentials. Experiential learning is a continuous process that 
transforms the impulses, feelings, and desires of concrete physiological and emotional 
experience into higher-order purposeful action (i.e. meanings). In that way the experiential 
learning style is purposeful and motivating (Kolb, 1984). By such transformations, 
experiences become the basis of new knowledge or of new practices. At the same time they 
engage us to continue or intensify the learning process. 


Positive experiences contribute actively to the seifs physiological and emotional wellbeing 
by eliciting positive emotions: e.g. emotions related to rewards, which are thus attractive. Our 
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use of experience design criteria is motivated by knowledge from positive psychology 
confirming that well-being and positive emotions promote cooperation between individuals 
(Seligman, 2000), are intrinsically motivating (Isen & Reeve, 2005), facilitates problem 
solving, broaden our scope of attention and modes of thinking (Frederickson, 2001; 
Frederickson & Branigan, 2005) and improves the understanding of the situation (Isen, 
Daubman, & Nowicki, 1987). 


In measuring differences in experience and the emotions generated by the three different 
learning designs the 10 criteria of “positive experiences”, introduced by (Jantzen et al., 2011), 
were used to develop the semantic differential questionnaire and guide the collection and 
analysis of interview and observation data. These criteria are firstly derived from theories on 
the psychology of experiencing (Jantzen, 2013) thus covering physiological, emotional, 
cognitive and social (e.g. identity issues) aspects: How and in which degree does the design 
for example promote emotional or cognitive aspects? What are its transformative qualities? 
Secondly, they stem from analyses of successful cases of experience design: Which structural 
features in the design do apparently have positive experiential effects? How does this 
particular design stand out from other designs, and which effects does this imply? 


These experience criteria therefore cover different dimensions of experiencing: 


■ Psychological aspects: whether the design is involving, relevant, interesting, and provide 
learning and understanding 

■ Structural aspects: whether the design is interactive, authentic, original, spontaneous and 
persuasive. 


Criteria 

Key questions 

Interactive 

Informants’ comments whether they feel an active part of the design: 
Do they feel that they are invited as co-players, co-producers or co¬ 
creators? 

Near 

Informants’ comments whether they find that the design “talks to 
them”: Does the design address their situation, their interests or their 
problems? 

Intimate 

Informants’ comments whether they feel obliged to participate: Does 
the design make them feel related, are they persuaded or convinced to 
become active or take responsibility? 

Authentic 

Informants’ comments whether they find the design authentic: Is the 
design sincere, true? 

Unique 

Informants’ comments whether they find the design original: Is it 
something that they have not experienced or encountered before? 

Involving 

Informants’ comments whether they feel emotionally involved: Is the 
experience exciting, relaxing or reassuring? 

Lively 

Informants’ comments whether they find that the design allow them to 
be spontaneous: Do they feel that the design encourage them to dig 
into the design? 
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Learning 

Informants’ comments whether they find that the design is supporting 
the learning process and the creation of experience: Does it challenge 
what they already know? Does it broaden their horizon? 

Understanding 

Informants’ comments whether they obtain understanding: Does the 
design facilitate the user’s comprehension of situations, intentions, 
potentials, etc.? 

Interesting 

Informants’ comments whether they find the design interesting: Is it 
providing something unexpected? Does it have their interest? Does it 
surprise? 

Relevant 

Informants’ comments whether they find the design relevant: Does the 
design relate to the existing mental concepts? 


Table 1: Sums up the 10 experience criteria and illustrates the key questions used to address 
the students’ experiences and feelings. The ten criteria also guided the observation of 
students’ emotional reaction in class and during project work. 


Some of these criteria express the degree in which the design involves its users emotionally 
and hedonically (Liveliness, Involvement). Other criteria cover the users’ physical 
(Interactivity) or personal (Nearness, Intimacy) engagement in the designs. And others again 
are cognitive (Relevance, Interest) or related to self-development and self-transformation 
(Learning, Understanding). Some criteria can be used to measure the design’s ability to 
motivate or persuade (Involvement, Liveliness, Intimacy), others for assessing its openness to 
active user participation and collaboration (Interactivity). Still others point to the surprising 
(Uniqueness, Interest) or sound (Authenticity, Relevant) qualities of the design. 

The 10 experience criteria are meant to cover the complexity of experiencing and the 
motivational direction and motivational force implied in having an experience. We therefore 
consider them to be useful metrics in measuring the relationship between the feelings 
generated by the three learning designs and in measuring how to promote student motivation 
for learning. 


SECTION III: REVIEW OF RELATED RESEARCH 

Finding a method to make teaching of computer programming more motivating for students is 
a global challenge. The following section describes some previous results obtained when 
introducing LEGO Mindstorms or a PBL approach in teaching computer programming. The 
three learning designs used in our study (section IV) build on these results. 
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Experiences with LEGO Mindstorms Robots 

Research on the use of LEGO Mindstorms robots in computer programming courses shows 
mixed results. One of the main advantages related with the use of LEGO Mindstorms is that 
these robots do not confine students to the constraints of a computer screen; instead they 
afford to teach computer-programming concepts using physical real world systems. In this 
vein, Garcia & McNeill (2002) stated that LEGO Mindstorms allowed students to control and 
manipulate computers in the real world making learning of introductory computer 
programming concepts more fun. In Lawhead et al. (2002) the authors argued that the robot is 
a real physical object, and as such very useful to teach concepts of object-oriented 
programming. Learning object-oriented programming is easier when students are offered 
physical objects that have the ability to "feel" their environment and react to it. This is in 
contrast with a traditional programming environment, which is often perceived by students as 
artificial or abstract. The robot can establish a direct relationship between programs and 
observable behavior, which is more satisfying for students as they can see the direct effect of 
their coding in robots, and get an immediate response if the robot does not behave as 
expected. This direct relationship between source code and its effect makes the testing phase 
really fun for the participating students (Lawhead et al., 2002). 

In the same vein, Anderson & McLoughlin (2007) mentioned that the lack of immediate and 
successful results that often comes with learning programming can have a negative impact on 
student motivation. This frustration can lead students to falling behind, failing the exams, and 
eventually dropping out of the study program. This situation is even more serious, given the 
lack of patience exhibited by current programming students. 

In Clibum (2006) it is described how LEGO Mindstorms were used in an introductory 
computer science course to introduce students to abstraction, algorithms, and problem 
solving. The author used the visual programming interface included in the LEGO Mindstorms 
software arguing that this allowed students to focus on problem solving rather than on 
learning the syntax of a programming language. This study recommended the use of LEGO 
Mindstorms as a tool to teach algorithms and foster student creativity. 

A research project made by Wong (2001) set out to study whether the use of LEGO 
Mindstorms robot activities could provide a more effective and motivational learning 
environment than the traditional Integrated Development Environments (IDE) which is 
common in most computer programming courses. The author included three weeks of LEGO 
Mindstorms activities on diverse levels of computer science courses. The author claimed that 
the students seemed to retain learned knowledge better in the LEGO sections than in the 
traditional ones. 
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On the other hand, there are also studies with non-favorable results. In Barnes (2002) the 
author indicates that it is impractical to use LEGO Mindstorms to teach an entire introductory 
programming course, mainly because of issues such as inconsistencies in the motor voltage 
and the possible confusing use of loop structures. Instead robots, he argued, may be used to 
support the learning of programming concepts in a traditional course setting. 

In Fagin & Merkle (2002), the authors reported the results of one year of experience in the use 
of LEGO Mindstorms activities in an introductory computer programming course. The aim of 
the study was to see whether the use of LEGO Mindstorms could improve student 
performance and determine the influence of robots in encouraging students to select computer 
science or computer engineering as a field of study. The study compared the results of more 
than 800 students on identical tests from both robotics and non-robotics-based laboratory 
sessions. The results were negative. Test scores in the robotics groups were lower than in the 
non-robotics groups, and using robots showed no measurable effects on the students’ choice 
of field of study. To explain these results the authors argued, that students in robotics groups 
must run and debug their programs on robots during assigned lab times, and therefore were 
deprived of time for reflection and of the compilation-run-debug cycle outside the classroom 
environment, which is an important part of the learning process. This also drastically reduces 
the amount of time available for reflective thinking on non-trivial projects given to students 
over several days. 

Similarly, McNally et al. (2006) concluded that there are logistical and pedagogical 
disadvantages in the use of robots. As logistical disadvantages the author outlined the costs, 
arguing that it is too expensive to provide each student their own robot which implies that 
every student-experimentation are limited to the available lab time in class, and this is 
insufficient to promote open experimentation with the robot. Regarding the pedagogical 
disadvantages, LEGO Mindstorms robots limit the scope of object-oriented concepts to which 
students can be exposed, because the robot did not support the exploration of concepts such as 
polymorphism or the interaction of multiple classes and objects. Other pedagogical 
disadvantages are related with the robots operating in a continuous world. This means, for 
example, inconsistency of robot movements due to differences in battery power, and the need 
for frequent calibration of sensors to respond to the changing nature of the physical 
environment. The authors argue that while the skills learned to program in a continuous 
environment are valuable and useful, they are not essential in the curriculum of computer 
science, and as such should not be the focus of the introductory students’ experience. 

Especially the practical time issues that diminish the time for reflection and limit the 
possibility of introducing a broad set of programming concepts, has caused debate on how to 
include activities with LEGO Mindstorms robots in introductory programming courses. 
Despite this, its use has become increasingly common at colleges and universities, because it 
is assumed that the use of these robots contributes to motivating students due to the close 

87 



M. Lykke, M. Coto, C. Jantzen et al 


JPBLHE: VOL. 3, No. 2, 2015 


relationship both to real-life problems and between programming and observable behavior of 
the robot. Overall, the use of robots provides a holistic, cognitive as well as embodied feel for 
programming. 

We have planned the PBL-LM learning design learning based on these findings by integrating 
theoretical lectures about programming theory in the project work when relevant, and by 
providing one robot per group throughout the programming course. 

Experiences with PEL approaches 

Problem-based learning (PBL) is a student-centered approach to learning, in which students 
learn through the process of solving an open-ended problem. PBL builds on constructivist 
principles, involves active learning and promotes collaborative learning (ACM, 2013; Prince 
& Felder, 2006; Hissey, 2000). The method strives to resemble a work-based scenario, either 
in the exploration and definition of a problem or as a simulation of a real-life project with 
more than one way to solve the problem or to implement the solution. Students work in small 
groups with the teachers as a supervisor or facilitator rather than a teacher. The method has 
the potential to achieve a higher motivation and greater responsibility in the learning process 
because students learn to be more independent in their approach instead of relying totally on 
teachers (Dirckinck-Holmfeld, 2002; Loyens, Joshua, & Rikers, 2008; Isen et al., 1987). 

Problem-based learning encourages students to face real problems as a starting point for the 
acquisition and integration of new knowledge (Prieto, 2006). The approach promotes the 
development of skills such as problem solving, decision-making, teamwork and 
communication skills. These characteristics are particular useful in computer engineering. The 
ability to solve problems is vital in the discipline and many of the activities of professionals in 
computer engineering are framed in the development of projects. Accordingly ACM (2011) 
identifies a set of skills that future graduates must have, such as problem solving, efficient 
communication, effective collaboration, professional responsibility and the capacity of 
lifelong learning. 

The effectiveness of PBL versus lecture-based teaching has been analyzed in several studies 
in the higher education context. The results are contradictory. According to Kinnunen & 
Malmi (2005), the results favor one or the other depending on whether the emphasis in 
learning is on the acquisition of factual knowledge or on self-directed learning skills, social 
skills and motivation. PBL may increase skill levels, but may result in poorer performance on 
traditional test subjects and it could also be stressful for students. Therefore some PBL 
learning designs include lectures, exercises or other pedagogical activities. 

In Nuutila et al. (2005) the researchers identified a significant decrease in the dropout rate in a 
study which introduced PBL in introductory programming courses. The authors argued that 
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in addition to learn programming students acquire skills in collaborative work, independent 
studying and communication. In the same vein, in a study on introducing PBL to teach 
theoretical concepts from computer science (Hamalainen, 2004), the author concludes that the 
dropout and failure rates decrease when students follow a PBL approach compared to a 
conventional one. Furthermore, the author reports a greater commitment of the students to the 
PBL course in comparison with a traditional one. 

Difficulties in using the PBL approach have also been identified. PBL involves a cultural 
change, both for students and teachers. In general, students are used to lecture-based methods 
of teaching, which promotes students to adopt a passive attitude and casts the instructor in the 
role of expert. Other problems are related to the main characteristics of PBL: problems as 
stimulus for learning, tutors as facilitators and group work as stimulus for interactions 
(Dolmans, De Grave, Wolfhagen, & van der Vleuten, 2005). In some learning environments 
students are confronted with too well structured and closed problems. In this case, problems 
are too simple to challenge students to construct knowledge actively. Another aspect 
hindering the PBL learning process is a too dominant or too lenient supervisor, which may 
provoke tension and conflict in groups leading to lack of commitment and student 
absenteeism (Dolmans et al., 2005). Regarding group work, some groups tend to be 
dysfunctional showing lack of cohesion and poor motivation, which obstructs the 
collaborative nature of learning (Kinnunen & Malmi, 2005; Dolmans et al., 2005). According 
to Dolmans et al. (2005), it is necessary to conduct further research to identify how PBL can 
stimulate students towards more constructive, self-directed, collaborative and contextual 
learning. 

We based the development of the PBL learning designs in our study on these findings. We 
related the theoretical concepts to real-life problems in the lectures and developed three broad 
project topics that the students utilized to formulate specific problems for their project. We 
divided the available lecture hours into two parts: a) lectures combined with small lab 
exercises and b) independent, student-led project work. 


SECTION IV: DESIGN OF THE STUDY 

We have seen that robots have the potential to engage students in the learning of computer 
programming. In addition our assumption was that the potentials of learning with robots are 
further increased when this technology is combined with PBL that supports a broad 
collaborative learning process and allow the presentation of more programming concepts and 
provide time for group discussion. 

The study utilized data from students enrolled in the course EIF200: Introduction to 
Programming during the first semester of 2013. The course taught the basic principles of 
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object-oriented programming and lasted 16 weeks with a student workload of 8 hours per 
week. 

The research design for the study was an experimental, controlled comparison study, which 
compared three learning designs: (1) a problem-based learning (PBL) design; (2) a 
combination of PBL and LEGO Mindstorms (PBL+LM) learning design; and (3) a traditional 
teaching design (control group). 

The study involved 15 groups of students and included a total of approximately 300 students 
and 12 faculty members. Each learning design was used for 5 groups. Each group in the 
Control and PBL groups consisted of a maximum of 25 students, and the groups in the 
PBL+LM design had 20 students. 

The study used 30 LEGO Mindstorms sets that were donated by The LEGO Loundation in 
Denmark. All students were included in the study with the exception of students who dropped 
the course and stopped coming to class. 


A. The three learning designs 


The three designs have several activities in common, as showed in table 1, but the PBL and 
PBL+LM learning designs have been designed according to the basic PBL principles 
supporting free, continued development of real world problems, process-oriented interaction, 
collaboration between students and professors, interdisciplinary problem-solving, self- and 
peer assessment, and a dynamic curriculum (Newman, 2005; Savory, 2006). Lectures do not 
have any weight on student scores, wherefore they do not appear in the table. 



Control 

PBL 

PBL+LM 

Problem-based project 

10% 

20% 

20% 

Learning activities (homework and 
quizzes) 

25% 

15% 

15% 

Attitudinal evaluation 

5% 

5% 

5% 

Exams 

60% 

60% 

60% 

Total 

100% 

100% 

100% 


Table 2: Distribution and evaluative weight of learning activities per learning design (in 

percentage) 


As shown, the PBL and PBL+LM learning designs have the same distribution of workload. 
The main difference with the learning design for the Control group concerns the learning 
activities and the project. The increase in the learning activities (homework and quizzes) was 
to compensate for the minor workload assigned to the project work. In all three designs the 
students took three exams, which together weighed 60% in the final grade. 
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Regarding the teaching approach, the participating professors in the PBL and PBL+LM 
learning designs introduced small problems to explain the various course topics (for example 
loop structures and arrays) while the professors in the learning design for the Control group 
consisted of teacher-led presentations of the topics either using the blackboard or a power- 
point presentation. In addition, the PBL and PBL+LM designs offered several challenges to 
the students throughout the semester to promote collaborative learning. In order to foster 
autonomy and responsibility for their own learning process the students in those learning 
designs were confronted with self-assessment and peer assessment strategies for each of the 
collaborative tasks (Rios, 2007). 

In the case of the PBL and PBL+LM learning designs, the students were put in groups of 4-5 
persons to work on the project. They had to choose from three different project topics 
formulated by the professors. These topics were described in an open-ended manner, so the 
student groups had to decide on the definition of the problem and the way to implement it. 
The projects in the PBL learning design addressed the use of bi-dimensional arrays while the 
projects with the PBL+LM learning design dealt with challenges for the robot, e.g. to collect 
trash. In the case of the learning design for the Control group there was only one project with 
a very detailed and structured description, leaving little room for independent development of 
the project. 

All groups in the PBL+LM learning design participated in a five-week sequence of lab 
activities using LEGO Mindstorms Robots. Construction of the robots was done in the first 
week. Robots were used to introduce selective (if-then-else) and iterative structures (while, do 
while and for). During the lab sessions the students worked in groups of 4 to 5 members. The 
C++ language was used for programming the LEGO robots. The decision to use C++ and not 
the visual programming interface included with LEGO Mindstorms software was to have all 
three learning designs using the same object-oriented programming language. The software 
used during the LEGO lab sessions was Microsoft Visual Studio 2010. Each group had their 
own robot to practice the lessons. In addition to the lab time the students could work with the 
robot in their own time but without taking the robot outside the university premises. 


B. Data collection methods 

User experience evaluation means investigating how a person senses and responds to a 
product, design, event or service (Vermeeren et al., 2010). It includes all the users' emotions, 
beliefs, preferences, perceptions, physical and psychological responses, behaviors and 
accomplishments occurring before, during and after use. The evaluation of user experiences is 
complicated by the fact that experiences are subjective, context-dependent and dynamic over 
time (cf. section II). They are subjective because they rely on the mood, knowledge and 
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momentary interests of the user. They are context-dependent in being influenced by 
circumstances in the immediate surroundings (weather, noise, accessibility etc.) as well as by 
larger social issues and cultural agendas. They are dynamic because new experiences relate to 
older ones and because memory transforms the quality and value of a past experience. 

To study the learning experience our research used mixed methods comprising qualitative as 
well as quantitative methods: 

■ Study 1 : A semantic differential questionnaire was developed to examine the students’ 
connotative perceptions of and attitudes to the learning design. 

■ Study 2 : The walk-along method was used to obtain opinion data and sensory information 
on the learning experience. This method consisted of a combination of interviews and 
observations while the student groups were working actively on their projects 
(Kusenbach, 2003;Lykke & Jantzen, 2013). 

■ Study 3 : Non-participant observations of classroom interactions were made to obtain 
insight into students’ behavior and emotions while being taught. 

■ Focus group interviews were conducted at the end of the empirical studies to validate the 
findings from the three other forms of data collection. 

The walk-alongs were planned to last an hour for each project group, and consisted of 3 steps: 
1) an introduction to the procedure; 2) observation of the project work while walking-along; 
and 3) follow-up interviews primarily to get demographic data about age and programming 
experience. The students and the walk-along facilitator met outside the classroom 
immediately after lecture. After a short introduction to the research project and the walk-along 
methodology, the facilitator walked along with the students, firstly to find a location for the 
group work, later to participate in the project work. On the way the PBL+LM groups picked 
up the LEGO Mindstorms tool box at the janitor’s office. All groups had problems finding a 
place to work. Two groups worked at the library, two groups in the outdoor patio, and the two 
PBL+LM groups in a computer room with small computer tables and limited floor space for 
working on and with the robot. During these walk-alongs the students were instructed to act 
and work as usual. The facilitator observed the group work, took notes, especially about the 
students’ interactions and mood, and asked clarifying questions about the students’ emotions 
and experiences with the project work and collaboration. The 10 criteria of the “positive 
experience” guided the observation and questioning. After an hour the facilitator closed the 
walk-along by collecting demographic information. All walk-alongs were taped. Immediately 
after the walk-alongs the facilitator made a summary of the course, summing up the students’ 
way of working focusing on the atmosphere and the emotional signature of the project work. 

The non-participant observation took place during lectures in the classroom. The observer 
was briefly introduced and placed at the back of the room. She took written notes on the 
course of events and the atmosphere in class. These observation studies were also based on 
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the 10 criteria of “positive experiences”. 

Findings from the questionnaire survey, walk-alongs and non-participant observations were 
discussed in three focus group interviews, one for each of the learning designs. The focus 
groups aimed at elaborating the understanding of the central research themes: students’ 
experiences, learning outcome, personal development and collaboration between students. 
Ten students from each of the three learning design groups participated in these focus group 
interviews. 


SECTION V: PRESENTATION OF RESULTS 

In the following section results from the three studies will be presented and discussed. 
Findings from the focus group interviews will be included in our discussion (section VI). We 
start by summing up results from the semantic differential questionnaire that provide a simple, 
quantitative picture of the students’ attitude and feelings about the learning designs. We use 
opinion, 93avourabl and sensory data collected during the walk-alongs as well as data from 
our non-participant observations to discuss the findings. 

229 students filled in the questionnaire, 70 students from the control design, 86 students from 
the PBL design, and 73 students from the PBL+LM design. 6 project groups participated in 
the walk-alongs, 2 for each of the three learning designs, a total of 21 students. Non¬ 
participant observations were made in the classroom during lectures, 2 lectures for each 
learning design: i.e. 6 lectures involving a total of 118 students. 

In the questionnaire survey there was a total of twelve pairs of opposite adjectives for the 
students to consider. The adjectives were related to negative or positive emotional states that 
one might have experienced during the course: 

■ Sad vs. Happy 

■ Annoyed vs. Comfortable 

■ Dissatisfied vs. Satisfied 

■ Melancholic vs. Delighted 

■ Despairing vs. Optimistic 

■ Bored vs. Stimulated 

■ Stressed vs. Relaxed 

■ Calm (tranquility) vs. Excited 

■ Slow vs. Hectic 

■ Nervous (anxiety) vs. Safe 

■ Sleepy vs. Lively 

■ Insignificant vs. Interested 
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They were placed adjacently on a scale with boxes that represented intermediate values. The 
students were told to put their mark in accordance to the box that best fitted their feelings 
during the lectures; the closer to the respective adjective the stronger the feeling. The students 
were also asked to provide supplementary comments about the respective course activities, in 
particular if they had experienced something interesting, challenging, or motivating. The 
method provided a semi-objective evaluation of the students’ emotional responses to their 
respective introductory course; control group, PBL, and PBL+LM. It was made clear to the 
students that it was not a test of their performance. 

For analytical purposes we categorized the replies to the questionnaire survey on a nine-point 
scale ranging from -4 over 0 to +4 between the pairs of adjectives; a negative value laden 
adjective and a positive value laden adjective such as the aforementioned “happy/sad”, 
“bored/simulated”, and so on. 

Upon this categorization we divided the nine-point scale into three parts. -4 to -2 were the 
very negative replies, -1 to +1 were the neutral replies, and +2 to +4 were the very positive 
replies. We then calculated the mean value of each of these parts, cf. table 3. The mean values 
made it possible to illustrate the differences in a radar chart, as can be seen in fig. 1. A 
negative mean value connects to a negative value laden adjective, and the reverse goes for 
positive values. Thus, in Table 3 the mean value 1,6 indicates an average feeling of sadness 
for the Control group and the values 2,0 and 2,2 an average feeling of happiness for the PBL 
and PBL+LM groups. 

In general, there were only slight differences in the students’ feelings, and the differences 
were triggered by a small set of students, between 1-30 students out of approximately 80 
students, depending on the differential. We will now present these differences in the 
following descriptions. 


Variable 

Control 

PBL 

PBL+LM 

Sad/Happy 

1,6 

2,0 

2,2 

Annoyed/Comfortable 

1,9 

2,3 

1,9 

Dissatisfied/Satisfied 

1,7 

1,9 

1,5 

Melancholic/Delighted 

2,0 

2,1 

1,8 

Despairing/Optimistic 

1,9 

1,4 

1,7 

B ored/S timulated 

2,0 

1,8 

1,9 

Stressed/Relaxed 

-0,1 

0,6 

-0,2 

Calm/Excited 

U 

U 

U 

Slow/Hectic 

0,6 

0,3 

0,3 

Nervous/Safe 

-0,1 

0,9 

-0,1 

Sleepy/Lively 

U 

0,7 

1,0 
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Insignificant/Interested 

2,9 

2,7 

2,8 


Table 3: Mean values of emotional scales 


In the following subsections we will go into more detail on the semantic differentials of each 
of the three learning designs. For this purpose we have given each number on the scale its 
own color respective to the value it represented. The warm colors represented the negative 
value laden adjectives of each of the 12 pairs of adjectives. Reversely, the cold colors 
represented the positive laden adjectives. The size of each of the color-coded bars represented 
the amount of replies connected to that given value. This allowed us to look for overall 
patterns between the three learning designs with ease. 



For the practical purpose of reading the graphs by color, we removed the numbers of how 
many students replied a certain value, which were situated in the middle of the color-coded 
bars. We will instead present the relevant numbers in the following descriptions. Also note 
that one of the pairs of adjectives “slow/hectic” can have multiple meanings, since “hectic” 
for instance may not be a positive feeling in all given situations, even though it is connected 
with the positive color coding in our graph. In general, there were only slight differences in 
the students’ feelings, and the differences are triggered by a small set of students, between 1 - 
30 students out of approximately 80 students, depending on the differential. 

E. The Control Group 

By looking at the color-values of the control group in fig. 2, we can see the students felt 
positive about the learning experience, but some also felt stressed and nervous. A small 
amount also felt bored and sad, but more or less everyone agreed that the course was 
interesting, cf. no negative colors at “insignificant/interested” in fig. 2. 
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Control Group 

N = 70 


■ -4 ■ -3 ■ -2 -I 0 I 2 ■ 3 ■ 4 


Sad / Happy 
Annoyed / Comfortable 
Dissatisfied / Satisfied 
Melancholic / Delighted 
Dispairing / Optimistic 
Bored / Stimulated 
Stressed / Relaxed 
Calm / Excited 
Slow / Hectic 
Nervous / Safe 
Sleepy / Lively 
Insignificant / Interested 



Fig. 2: Semantic differentials for the Control Group students. 


The number of students who replied they felt between -2 to -4 values of stress was 20 out of 
68, which means that at least 29% of those who replied felt some amount of stress. Likewise, 
the number of students who felt some amount of nervousness was 33% of the 70 students who 
replied. 14% replied they were sad and 6% replied they were bored. 


The observational data confirms the findings. In the control groups the professors used either 
the blackboard or a power point file to present the logic of the programming in a sequential 
manner. The professors were relaxed, interacted in a living and cheerful way with the 
students, and involved them in the teaching. Some few students took directly part in the 
dialogue answering questions from the professor, but the majority of the students followed the 
teaching with attention and interest, but passively. Judged on their expressions some students 
seemed distracted. Some seemed to have trouble understanding the lecture, but only few 
consulted the professor with questions. Some students small-talked during the lecture 
disturbing the concentration of other students. Both active and passive students used their 
mobile during class. A small group of students discussed the programming problem and 
solution among them independently of the professor, but most students left initiative and 
organization of the learning process to the professor. 


The Control group students made a small project as part of the course, which was observed 
during the walk-alongs. Compared to the classroom teaching the control group students were 
more involved and enthusiastic in the project work. They tried out solutions, consulted the 
Internet for information, and discussed solutions. However, as in class, some few specific 
students took the lead of the project work, and in general the students expressed frustration 
being without the guidance of the professor. 
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B. PBL 

Looking at the color-values in the PBL course in fig. 3 we see a similar pattern of overall 
positive experience with some amount of stress and nervousness. The number of students who 
felt happy seems to be a bit higher than in the control course. Some students also found the 
PBL course to be somewhat insignificant. _ 

PBL 

m-4 ■ -3 ■ -2 ■-P =a7 0 ■! 12 13 u4 



Fig. 3: Semantic differentials for the PBL group students. 

The number of students who replied they felt between -2 to -4 values of stress was 24 out of 
86, which means that at least 28% of those who replied felt some stress. The total of students 
who felt somewhat nervous was 16% out of the 86 students who replied. 

Only 1% felt sad and 3% felt bored, which is an improvement when compared to the Control 
groups’ course. 

Looking at the number of students who felt the course was somewhat insignificant, 3 students 
out of a total of 87 replied they felt between -2 to -4 values. This amounts to 3% of the total 
population in PBL. 

Again the observation and walk-along data confirm the questionnaire findings. Both PBL 
classes were held in a form continually switching between a short lecture, an instruction by 
the professor, and group work. In total there were three small group work sessions during the 
observed class. As in the control group the tone was lively and joyful with a close contact 
between the professor and the students. The students paid attention to the professor and 
participated with comments and questions. There was laughing, and the students expressed 
involvement and interest. Not all students participated actively, but only one group of boys in 
one of the PBL classes used their mobiles and small-talked. During the project work, the 
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students worked very interactively, exchanging and discussing ideas, clearly trying to apply 
concepts from the lecture and using the course terminology. There was a lively, relaxed and 
humorous atmosphere with friendly competitive and reciprocal teasing - e.g. in regard to who 
knew the proper terms or who found the best solution to the problem. Many of the discussions 
was about the “best” or “right” way to solve the problem. No specific plan or task 
organization was made for the work. The students did not divide the tasks between them, all 
contributed continually. Most group members participated actively. Generally, the students 
highlighted the realistic tasks that they found enriching and very motivating. 

PBL + LM 

N= 73 

■ -4 ■ -3 ■ -2 ■ -1 0bIb2b3b4 

Sad / Happy 
Annoyed / Comfortable 
Dissatisfied / Satisfied 
Melancholic / Delighted 
Dispairing / Optimistic 
Bored / Stimulated 
Stressed / Relaxed 
Calm / Excited 
Slow / Hectic 
Nervous / Safe 
Sleepy / Lively 
Insignificant / Interested 

Fig. 4: Semantic differentials for the PBL+LM group students. 

C. PBL + LEGO Mindstorms 

The color-values in the PBL+LM course show a similar pattern as with the other two course 
types; stress and nervousness being the dominant negative states of feeling. The number of 
students who replied they felt between -2 to -4 values of stress was 26 out of 72, which means 
that at least 36% of those who replied felt some amount of stress. The total of students who 
felt some degree of nervousness was 28% out of the 71 students who replied. 

None of the 68 students who replied felt sad, while 3% of the 72 who replied felt bored. One 
student felt the course to be somewhat insignificant. He replied that he felt a value of -3. 

The PBL+LM class consisted of project work, with only a short introduction to the 
programming subject by the professor. The students were very concentrated in their work, 
continually switching between programming the software and trying out the results with the 
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robot. In general all students were involved. The students worked very structured and divided 
the work into tasks or roles, e.g. adjusting the algorithm, consulting the literature or the 
teacher, operating the robot, or repairing the robot. In most groups one or two of the students 
took the lead and organized the work; typically the ones who operated the computer or the 
robot. Thus, all students were interested and participated interactively. All felt related to the 
work, but the students participated with various degrees of involvement and liveliness. Some 
expressed frustration when they repeatedly had to make small algorithmic changes in order to 
have the robot circle or turn. The professor worked around the class and participated actively 
in the work. Sometimes the students had to wait for help. 

For the PBL+LM students the start of the project work was slow and annoying because the 
students had to pick up the robot at the Support and Development Unit, find a room, and build 
the robot. The groups also had problems and used much project time to set up the 
communication between the computer and the robot. In addition, they had problems finding 
floor space to try out the programming on the robot. However, the atmosphere was good. The 
students were very interested in the task. They sat close to each other while working, 
discussed solutions, and tried out solutions. They were very engaged and helped each other. 

We have now presented the students’ emotional responses to the three learning designs and 
will now compare how these designs score on negative emotions. Our assumption is that 
“positive experiences” relate to lower scores on this set of emotions. Lower scores are thus 
assumed to be 99avourable to learning. 

D. Stress & Nervousness 

By illustrating the semantic differentials in the color-coded graphs we have revealed that there 
is a clear pattern of stress and nervousness in each of the three learning designs. To see how 
much the courses differ, we will now compare the findings from each of the three learning 
designs. 



Fig. 5: Stress values across learning designs. 
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Comparing the percentages of students who feel stressed in each learning design, we find that 
the PBL+LM course scores the highest, cf. fig. 5. Interestingly, the PBL course, which only 
differs slightly in its approach compared to PBL+LM, scores the lowest of the three. 



Fig. 6: Nervousness values across learning designs. 

Looking at the percentages of students who feel some degree of nervousness, once again, the 
PBL course has the lowest score, cf. fig. 6. It is also somewhat surprising that the Control 
group course scores the highest percentage, considering the novelty of the learning designs in 
both the PBL and the PBL+LM courses. 

E. Sad, Bored & Insignificant 

When comparing the percentages of students who feel sad in each learning design it is quite 
noticeably the Control group course that scores the worst, cf. fig. 7. With 14% feeling 
unhappy there is clearly a difference compared to PBL and PBL+LM, which scores 1% and 
0%. Still, a simple glance at the color-coded graphs reveals that the majority of the students 
feel happy in the respective course types. 



Fig. 7: Values for sadness across learning designs 

Looking at the percentages of students who feel bored, once again the Control group course 
scores the worst, but only with a three percent increase compared to PBL and PBL+LM, cf. 
fig. 8. As in the case of feeling sad, the color-coded graphs reveal that the majority of the 
students felt stimulated in their respective course types. 
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Fig. 8: Values for boredom across learning designs. 

The amount of students who felt their course type was insignificant is only 1% in the 
PBL+LM course and 3% in the PBL course, cf. fig. 9. Interestingly, there is no one in the 
Control group who felt their course was insignificant. 



Fig. 9: Values for insignificance across learning designs. 

The focus group interviews provided additional explication for the findings. Working with the 
robots was very motivating and interesting for the students. The students considered the 
possibility of working with robots as an interesting, fun and exceptional opportunity. They 
liked to work interactively with the programming, liked the trial-error learning style that was 
essential for the robot work. However, the students experienced a lack of theoretical 
knowledge that could guide them through the trial-error process. It provided a feeling of 
insecurity and doubt - e.g. “do we learn what we should”, “do we obtain sufficient theoretical 
programming knowledge”. In addition, the students felt a time pressure, because they used 
much time to pick up and assemble the robot and find a place to work. They were also 
frustrated because they did not have sufficient space or appropriate physical conditions to try 
out the robots. 

The Control groups were satisfied to have a professor that systematically presented 
programming concepts and guided the problem-solving process. It made them feel safe about 
the learning outcome. They also liked the one small project work. However, when they had to 
plan the work themselves, they felt unprepared and insecure about the process and the group 
work - questions raised were of the type: “How do we approach the problem?” “How are we 
to organize the work?” “How are we supposed to work together?” 
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The PBL groups enjoyed the dynamic interplay between lecture and group work and the 
collaborative interactions with the professor in class. The interactive learning style in class 
provided a feeling of security, and the students felt comfortable and prepared for the project 
work. The PBL students highlight the trial-error learning process and they experienced the 
collaborative interactions between individual considerations, ideas and experience from co¬ 
students and guidance from the teacher as interesting, challenging and fun. 


SECTION VI: DISCUSSION 

The aim of this study was to leam about the influence of learning designs on students’ 
emotions and on their intrinsic motivation for learning. Based on the assumption that 
experiential value is derived both from hedonic experiences and from engagement (Higgins, 
2006), the study focused on the students’ emotional response to and their feeling engaged in 
these designs. Ten criteria of “positive experiences” were used to study the emotional 
qualities of the designs. These criteria cover different dimensions of the learning experience - 
whether the students feel that they learn something, derive pleasure, comfort or inspiration 
from this learning, whether they work in an interactive and collaborative environment, feel 
motivated, and feel responsible for their own learning process etc. The use of experience 
design criteria is motivated by knowledge from positive psychology confirming that well¬ 
being, positive emotions and self-activity promote cooperation, are intrinsically motivating, 
broaden the scope of attention and thinking and facilitate problem solving. 

Our research shows that the learning designs influence the students’ physiological and 
emotional wellbeing (i.e. their emotional responses) as well as their active engagement in the 
learning process. Working with the robots was experienced as engaging by the students. The 
students considered the possibility of working with robots as an interesting, fun and 
exceptional opportunity. They liked to work interactively with the programming, felt 
motivated by the trial-and-error learning style that was essential for working on and with the 
robot. Nonetheless, they also expressed frustration and de-motivation when they repeatedly 
had to make tiny changes in the programming code. They felt that insignificant programming 
details shifted the focus away from more general programming principles. Additionally, 
students experienced a sense of insecurity and doubt about the learning outcome, mainly due 
to a perceived lack of theoretical knowledge to guide them through the trial-and-error process. 
Furthermore, they felt a time pressure because whenever they were required to work with the 
robot, they needed to pick it up, to assemble it and to find a convenient place to work. All 
these activities took much time and distracted them from the programming work. The strength 
of PBL+LM learning design is the project work and a high degree of interaction and 
collaboration, whereas the students miss the freedom to develop and frame the purpose of the 
project. Likewise, the nature of the robot work force the students to divide tasks between 
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them, thus narrowing the students’ learning experience. At the same time, the inherent 
limitations in developing real world programming tasks by using the robot is a challenging 
obstacle for the learning design. 

The PBL groups enjoyed the interactive collaboration with the professors and the dynamic 
interaction between lectures and group work, which took place in the classroom. Contrary to 
the PBL+LM students the PBL students did not divide the tasks between them. No specific 
plan or task organization was made for the work, most PBL students contributing continually 
and interactively, thereby potentially engaging each student in an optimal way and allowing 
them to obtain a broad learning experience. In the same vein, the interactive learning style in 
class provided a feeling of security, and the students felt comfortable and prepared for the 
project work. The PBL students highlighted the trial-and-error learning process and the 
collaborative interaction between individual considerations, ideas and experiences from co¬ 
students and guidance from the teacher. The collaborative work was challenging, but 
primarily experienced as interesting and fun: i.e. as emotionally rewarding. As it might be 
expected from previous literature, the PBL-approach provided a productive environment for 
experiental learning. In the present set-up, however, the degree of freedom to develop the 
problem and to plan the problem-solving process may still be improved providing more 
challenges and skills to the students. 

The Control group primarily received teacher-led lectures with a small, well-defined project 
as part of the course. The majority of the students followed the teaching with some degree of 
attention and interest, but they did it passively. Only some few students took directly part in 
the dialogue answering questions from the professor. The overall intensity of the motivational 
force appeared significantly lower than in the two other designs. Generally speaking, the 
students left the organization of the learning process to the professor, seemed satisfied to have 
a professor that presented programming concepts and guided the problem-solving process. It 
made them feel safe about the learning outcome. 

Nevertheless, they appreciated the one small project work, felt motivated by the realistic 
problems and the collaboration with other students. However, when they had to plan the work 
themselves, they felt unprepared and insecure about the process and the group work - 
questions raised were of the type: “How do we approach the problem?” “How are we to 
organize the work?” “How are we supposed to work together?” The strength of the classroom 
learning design is its potentials for continuity and control of the learning outcome. However, 
the Control groups seem prone to a feeling of stress and nervousness. They did not feel 
prepared to take responsibility for their own learning and collaborative work. And they 
expressed frustration when being without the guidance of the professor to motivate their 
learning process. 


103 



M. Lykke, M. Coto, C. Jantzen et al 


JPBLHE: VOL. 3, No. 2, 2015 


Except for the semantic differential questionnaire that was filled out by 229 students, with a 
response rate of 69%, the study is small with 2 walk-alongs and 2 classroom observations per 
learning design. The findings would be stronger if walks had been conducted with all 15 
project-groups and if there had been walks at intervals during the whole learning period. Such 
approach would have provided a broader picture of the group work and would have allowed 
the researchers to gain more insight into the developments in the students’ work processes, 
their collaborative behavior, and their engagement in group work and programming theory. 
To compensate for this lack, the explicit aim of the follow-up focus group interviews was 
therefore to validate the findings and thus provide a broader picture of the learning designs. 
The focus group interviews that were carried out with both students and faculty (each group 
apart) allowed us to compare findings across project groups giving us with a more nuanced 
picture of the learning designs. 

We deliberately chose to carry out the walk-alongs at a point approximately at mid-point in 
the course. The idea was to meet the students when they had worked with the learning designs 
for some time, overcome unavoidable start problems and gained experience with these new 
ways of learning. 

The combination of quantitative, less nuanced information from the semantic differential 
questionnaire and the situational and detailed insight from the walk-along and observation 
studies gave a nuanced picture of students’ experiences. The data supplemented and 
supported each other pointing to the same findings. Specifically we found it useful to go from 
surprising results from the semantic differential data, e.g. telling that traditional class-room 
learning scored well on interest and low on stress, to the narratives and explanations in the 
walk-along conversations. 


SECTION VII: IMPLICATIONS 

Summing up, all three learning designs have their own set of advantages and challenges. The 
PBL-design seems best at stimulating collaboration, interaction, and emotional wellbeing. 
The robots in the PBL+LM-design are engaging and motivating, but also frustrating, due to 
the inherent limitations of the robot regarding project tasks, practical issues and insecurity 
about the learning outcome. The traditional class lectures provide security in terms of 
theoretical insight, but also provide stress and nervousness due to little or lacking experience 
with working actively and collaboratively. If we want happy, comfortable, delighted, and at 
the same time calm and lively students, none of the learning designs are completely 
satisfying. It is also clear that all of the students are motivated by working in projects, but for 
the robots to become an effective tool for motivation it is necessary to provide more 
theoretical knowledge about programming and to improve the project tasks and the conditions 
under which the robots are used in the course. 
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Concerning the use of LEGO Mindstorms robots the results underpin the importance of 
practical issues. There should be one robot per group, each group should keep their robot 
during the full project period, and appropriate physical environments with sufficient space for 
working with the robots should be provided. The idea of integrating and relating the 
theoretical lectures directly to the practical work with the robots worked well. However, it 
should be clear for the students how the small lectures relate to the overall curriculum. If not 
this way of lecturing may cause uncertainty whether “we learn what we should”. The 
students’ emotional response to traditional lectures shows the strength of close contact 
between students and lecturer. The lecturer provides security and also interest and motivation 
by prioritizing and structuring the theoretical subjects. PBL turned out to be the most 
motivating and engaging approach. However, the students expressed less interest and 
stimulation compared to traditional teacher-led courses. This is surprising. The findings show 
the importance of the guiding role of the teacher. This group of students, students for whom 
self-directed, independent group work was a new phenomenon, appreciated the teacher’s road 
map and directions. 
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